Optical tracking systems

ABSTRACT

A method for tracking the position and orientation of an ultrasound beam emitted from an ultrasound probe. The method comprises the steps of: storing a model of the visible object in a memory of an optical tracking system; calculating the position and orientation of the visible object; and calculating the position and orientation of the ultrasound beam by applying a known geometric relationship to the deduced position and orientation of the visible object.

[0001] The present invention relates to optical tracking systems. Moreparticularly, it relates to optical tracking devices allowing thelocation of certain objects to be calculated, which objects cannotthemselves be optically tracked.

[0002] For example, when an ultrasound probe is in use, possibly for theproduction of a three-dimensional ultrasound image, the ultrasound beamcannot itself be visually tracked, as it is invisible. In anothersituation, an ultrasound probe may be used for industrialnon-destructive testing. This will encounter the same problems ofinvisibility of the ultrasound beam.

[0003] The present invention therefore addresses these problems and aimsto provide optical tracking methods and apparatus for equipment, such asprobes, for which the locations of the points of measurement or thepoints of effect, such as ultrasound beams, are not directly measurable,but are in a known geometric relationship to a part of the equipmentwhose position and orientation can be measured.

[0004] The present invention accordingly provides a method for trackingthe position and orientation of an ultrasound beam emitted from anultrasound probe, the ultrasound beam bearing a geometric relationshipto a visible object, comprising the steps of: storing a model of thevisible object in a memory of an optical tracking system; calculatingthe position and orientation of the visible object; and calculating theposition and orientation of the ultrasound beam by applying thegeometric relationship to the deduced position and orientation of thevisible object.

[0005] The position and orientation of the visible object may itselfcomprise the steps of: estimating a position and orientation of thevisible object; generating an estimated image of the visible object inthe estimated position and orientation; comparing the estimated imagewith a video image of the visible object, and adjusting the estimatedimage by adjusting the estimated position and orientation, until theestimated image corresponds to the video image, thereby deducing theactual position and orientation of the visible object.

[0006] The calculated position and orientation of the ultrasound beammay be expressed in co-ordinates expressed with respect to an observer.The method may then further comprise the step of translating thecalculated position and orientation into a position and orientationexpressed with reference to an object under test.

[0007] The method may farther comprise a calibration step in which thegeometric relationship between the ultrasound beam and the visibleobject is deduced.

[0008] The visible object may comprise a visually distinct object, beingone of: a marker attached to a visible part of the probe; visualmarkings on the surface of the probe; or the shape and profile of theprobe itself.

[0009] A plurality of ultrasound beams may be simultaneously tracked.

[0010] In an embodiment of the invention, the probe is a scanningultrasound probe producing two-dimensional images, the ultrasound beamis planar and the visible object is a visible part of the probe,wherein: the probe is moved across an object to be scanned; the positionand attitude of the probe is measured and recorded as a function oftime; images produced by the probe are recorded as a function of time;and the recorded positions, attitudes and images are combined to producea three dimensional image of the scanned object.

[0011] A further probe may also be tracked, in which case the relativeposition and orientation of the ultrasound beam and the point of effectof the further probe may each be calculated.

[0012] In such an embodiment of the present invention, the further probemay be a biopsy needle. The method may then comprise moving theultrasound probe across a body to be scanned; measuring the position andattitude of the ultrasound probe; measuring the position and attitude ofthe biopsy needle; calculating the positions and attitudes of theultrasound beam and the tip of the biopsy needle; and displaying theposition of the biopsy needle on the image provided by the ultrasoundscanner.

[0013] These, and other, features, characteristics and advantages of thepresent invention will become more apparent with reference to thefollowing description of certain embodiments of the present invention,given by way of examples only, with reference to the accompanyingdrawings in which:

[0014]FIG. 1 shows a modelled image and a video image of a trackedobject; and

[0015]FIG. 2 shows possible differences between the modelled image andthe observed video image.

[0016] In order to track the location of the point of measurement or thepoint of effect of equipment such as probes, the position andorientation of a part of the equipment whose position and orientationcan be measured may be tracked. The required point of measurement oreffect can then be calculated if it bears a known geometric relationshipto the position and orientation of the part of the equipment whoseposition and orientation can be measured.

[0017] In an example, the position and orientation of a handle of abiopsy needle may be tracked, and the position and orientation of thepoint of effect, at the end of the needle, may be calculated accordingto the known geometry of the needle.

[0018] A three-dimensional video tracking system may be employed todetermine the position and orientation of a visible part of theequipment. An example of a suitable system is described in UK Patent GB2 246 261. The system described in that document involves the comparisonof a modelled image, based on a previously stored model of the trackedobject in conjunction with an assumed location and orientation of theobject, with a video image of the object. By calculating the differencesin position of various reference points on the model and thecorresponding points on the video image, the actual position andorientation of the tracked object may be calculated. As described inthat document, the modelled image may be adjusted until it matches thevideo image. Alternatively, as described therein with reference toaircraft navigation, the position (flight path) of the observer (videocamera) may be adjusted until the video image matches the modelledimage. By repeatedly comparing the images and adjusting the modelledimage, the location and orientation of the equipment may be tracked.

[0019] Other types of video tracking system may be employed in thesystem of the present invention, but the system used must be able to usedata from video imagery to continuously track, and calculate theposition and orientation of, visually distinct rigid objects for which3D models are available.

[0020] The visually distinct object tracked by a tracking system may be:a marker attached to the visible part of the equipment (such as aprobe); visual markings such as a printed pattern or symbol on thesurface of the equipment; or the shape and profile of the equipmentitself.

[0021] A probe may be powered, in which case a light emitting means suchas one or more LED's may be attached to it, and the light emitting meansmay be used as a marker for tracking the position and orientation of theprobe. Alternatively, an unpowered probe may be used, and ambient ordirected light may be employed to illuminate the object, to rendervisible the marker, markings or shape and profile of the probe.

[0022]FIG. 1 schematically shows a modelled image 1 and a video image 2of a handle of a biopsy needle. In the present case, the profile of thehandle itself is used as the marker. Four visible corners may be used asreference points, calculating intermediate reference points 3, 5 in themodelled and video images, respectively.

[0023] The orientation and dimensions of the modelled image is adjusteduntil it matches the observed video image. FIGS. 2(a) to 2(f) illustratevarious possible differences between the modelled image and the observedvideo image. These represent, respectively: (a) the true position of thehandle is lower than the modelled estimated position; (b) the trueposition of the handle is rolled to the left of the modelled estimatedposition; (c) the true position of the handle lies to the left of themodelled estimated position; (d) the true orientation of the handle ispitched up from the modelled estimated position; (e) the true positionof the handle is nearer than the modelled estimated position; and (f)the true orientation of the handle is yawed to the left of the modelledestimated position of the handle.

[0024] The differences between the locations of respective referencepoints 3, 5 in the modelled and true images are then reduced byrecalculating the modelled estimated image until the two images match.The location and orientation of the handle is then known, in threedimensions. Since the point of interest, that is, a point of measurementor point of effect of a needle or probe or the like, bears a knownthree-dimensional geometric relationship to the position and orientationof the handle, the position and orientation of the point of interest maybe simply calculated.

[0025] This will, however, provide the position of interest only in theco-ordinate system of the observer, that is, relative to the position ofthe video camera. It will not provide the position of the point ofinterest with relation to the object under examination.

[0026] For example, it is of limited use to know that the point ofmeasurement or point of effect is at so many millimetres in each of x, yand z directions from the camera, if one cannot also deduce which partof an object is being acted upon, for example, from which part of a bodyan autopsy needle is taking a tissue sample.

[0027] The optical tracking system used in the present invention mayalso track the position and orientation of the object under inspection,and may then use this tracking to locate the point of interest in theobject's co-ordinate system, as will be described in more detail below.

[0028] It may be necessary to employ more than one camera in the opticaltracking system, to improve measurement accuracy. In some embodiments,the optical tracking system may track more than one piece of equipment(e.g. probe), and may then calculate the relative positions andorientations of the probes.

[0029] While the present invention enables the orientation and locationof any type of equipment to be deduced, it has particular application tothe tracking of probes in medical and industrial testing environments.

[0030] The following types of probe could be used in embodiments of thepresent invention.

[0031] (i) Point probes, such as temperature probes, pH probes, needlesand syringes, which act at a single point. The system of the presentinvention is used to determine the location of the probe tip.

[0032] (ii) Axial probes, such as single angle ultrasound probes, whichact in one dimension, along a line emanating from the location of theprobe. The system of the present invention is used to determine thelocation of the line of the probe.

[0033] (iii) Planar probes such as scanned angle ultrasonic probes,which act in two dimensions, across a plane emanating from the locationof the probe. The system of the present invention is used to determinethe location and orientation of the plane of the probe.

[0034] (iv) Three-dimensional probes, such as CT or magneto-resonanceimaging (MRI) probes, act in all directions from the probe, but theorientation of the probe must be accurately known in order to make senseof the measurements provided by the probe. The system of the presentinvention is used to determine the location and orientation of thevolume covered by the probe.

[0035] Hence, the point of measurement or point of effect which is to belocated may be three dimensional (volume), two dimensional (plane), onedimensional (line), or without dimension (point).

[0036] The desired result of tracking a probe is usually that theposition of the point of measurement or point of effect may bedetermined in a desired co-ordinate system. A first stage in this is tofind the position of the point of measurement or effect in theco-ordinate system of the camera. To achieve this, a calibration must beperformed to find the geometric relationship between the point ofmeasurement or effect and the visually distinct object, so that themeasurement of the marker position can be used to find the position ofthe point of interest. A correlation must therefore be determinedbetween the three dimensional position of the visually distinct object,and the position of the point of measurement or point of effect, whichmay itself be a point, line, plane or volume, as discussed above.

[0037] For point probes and axial probes, calibration may be carried outautomatically, for example using a process in which the tip (point ofmeasurement or point of effect) of the probe is held fixed, and theprobe is moved around it, while the visually distinct object is trackedby the optical tracking system. From the measurements of the position ofthe visually distinct object, the location of the tip with respect tothe visually distinct object may be deduced.

[0038] With particular reference to the present invention, for axial andplanar ultrasound probes, calibration can be carried out by acceptedultrasound calibration techniques.

[0039] A registration process may need to be carried out, to find therelationship between the camera co-ordinate system and the co-ordinatesystem for the object under test, so that measurements made by thecameras can be reported in the object's co-ordinate system. Any knownpoint-based registration process may be used, whereby the positions ofknown points on the object are measured in camera co-ordinates, thesebeing matched to measurements of the same points in the objectco-ordinates, and the transformation between these sets of points isthen found. Typically, this operation would be carried out at the startof each test.

[0040] Alternatively, the transformation between camera coordinates andobject co-ordinates may be found opportunistically by measuring theprobe position throughout the test, whenever it is in contact with theinspected object. Surface reflection may be employed to determine whenthe probe is in contact with the surface. With knowledge of the positionof the active point of the probe with respect to the tracked visuallydistinct object, a model of the surface of the object under test can bereconstructed, which is then matched to a model of the object in theobject co-ordinate system (such as a CAD model) to calculate thetransformation between the two sets of co-ordinates.

[0041] Certain specific examples of implementation of the presentinvention, along with the particular advantages arising from theseimplementations, will now be discussed.

[0042] Three-dimensional (3D) ultrasound scanning is a growing field.Data is collected from a planar probe, and corresponds totwo-dimensional ultrasound scans, representing slices through theobject. The data from these slices is processed to give an overall threedimensional representation of the object. This representation may beviewed from any desired angle, to provide a much clearer impression ofthe structure of the object than would be possible from the slice dataalone. In order for this three-dimensional processing to be effective,the position and orientation of the ultrasound probe must be known atall times during data collection, so that the data can be reassembledinto 3D form. Many of the methods currently used for probe positionmeasurement are less than satisfactory, and an optical tracking systemaccording to the present invention may improve the efficiency of probeposition measurement, and so improve the clarity and reliability of thethree-dimensional images produced.

[0043] In an embodiment of the present invention, an ultrasound probe ismoved across the object to be scanned. An optical tracking system, suchas described in itself above, will measure the position and attitude ofthe ultrasound probe, and this will be recorded as a function of time.The two-dimensional ultrasound scans will also be recorded as a functionof time. This information will allow the three dimensional scannedvolume to be reconstructed as an image.

[0044] An ultrasound guided biopsy technique is known, for extractingtissue from a patient with minimal intrusion. An operator, usually aradiologist, scans the patient using ultrasound, views the site forbiopsy, and guides the biopsy needle to the correct position. However,it has been found difficult to see the biopsy needle on the scan. Thepresent invention may be applied to track both the ultrasound probe andthe biopsy needle, allowing the position of the biopsy needle to bedisplayed on the ultrasound scan image.

[0045] Operation of such a system will involve the following steps: theultrasound probe will be moved across the patient to produce a scanimage, while the biopsy needle is held in approximately the correctposition, as judged by the radiologist. An optical tracking system, asdescribed in itself above, will track both the probe and the biopsyneedle and measure their positions and attitudes. Using calibrations, asdiscussed above, for each of the probe and the biopsy needle, theposition of the needle axis and tip can be calculated in the scanco-ordinate system and displayed on the scan image as a graphicaloverlay. This allows the operator to guide the tip of the biopsy needleto the exact point required for sampling, regardless of whether thepatient changes position during the process.

[0046] Ultrasound scanning is used in industrial non-destructivetesting. Typically, an ultrasound operator manually records the positionof the probe when defects were found, and later calculates the defectlocation from knowledge of the probe characteristics, the timing of theultrasound signal, and the speed of sound in the tested material. In anembodiment of the present invention, an optical tracking system, asdescribed in itself above, is used to automate the recording process, soreducing errors and the time taken. Measurements of the probe positionmay be recorded by the optical tracking system, to ensure that theobject has been fully scanned for defects.

[0047] The operator will move the ultrasound probe across the objectunder test. The optical tracking system will track the probe, andmeasure its position and orientation. The operator will indicate when adefect is discovered, and this data will be recorded. Using the probecalibration, calculated as described above, the positions of defects canthen be found in the camera co-ordinate system. This measurement canthen be translated into the required location of the defect within theobject using the registration procedure described above.

[0048] While the present invention has been particularly described withreference to a certain number of specific embodiments, given by way ofexample only, the invention may be applied in various modified forms.

[0049] For example, while the invention may be applied to tracking theposition of features obscured by reason of their insertion into a body,the invention may also be applied to tracking the position of featureswhich cannot be directly tracked, either due to their being invisible;or their being obscured by other pieces of equipment; or simply becausethe feature in question cannot be clearly detected by a visual trackingsystem.

1. A method for tracking the position and orientation of an ultrasoundbeam emitted from an ultrasound probe, the ultrasound beam bearing ageometric relationship to a visible object, comprising the steps ofstoring a model of the visible object in a memory of an optical trackingsystem; calculating the position and orientation of the visible object;and calculating the position and orientation of the ultrasound beam byapplying the geometric relationship to the deduced position andorientation of the visible object.
 2. A method according to claim 1wherein the step of calculating the position and orientation of thevisible object itself comprises the steps of: estimating a position andorientation of the visible object; generating an estimated image of thevisible object in the estimated position and orientation; comparing theestimated image with a video image of the visible object; and adjustingthe estimated image by adjusting the estimated position and orientation,until the estimated image corresponds to the video image, therebydeducing the actual position and orientation of the visible object.
 3. Amethod according to claim 1 or claim 2 wherein the calculated positionand orientation of the ultrasound beam is expressed in co-ordinatesexpressed with respect to an observer.
 4. A method according to claim 3,further comprising the steps of translating the calculated position andorientation into a position and orientation expressed with reference toan object under test.
 5. A method according to any preceding claimfurther comprising a calibration step in which the geometricrelationship between the ultrasound beam and the visible object isdeduced.
 6. A method according to any preceding claim wherein thevisible object comprises a visually distinct object, being one of: amarker attached to a visible part of the probe; visual markings on thesurface of the probe; or the shape and profile of the probe itself.
 7. Amethod according to any preceding claims wherein a plurality ofultrasound beams are simultaneously tracked.
 8. A method according toany preceding claim wherein the probe is a scanning ultrasound probeproducing two-dimensional images, the ultrasound beam is planar and thevisible object is a visible part of the probe, wherein: the probe ismoved across an object to be scanned; the position and attitude of theprobe is measured and recorded as a function of time; images produced bythe probe are recorded as a function of time; and the recordedpositions, attitudes and images are combined to produce a threedimensional image of the scanned object.
 9. A method according to anypreceding claim wherein a further probe is tracked, and the relativeposition and orientation of the ultrasound beam and the point of effectof the fixer probe is calculated.
 10. A method according to claim 9wherein the further probe is a biopsy needle, and wherein: theultrasound probe is moved across a body to be scanned; the position andattitude of the ultrasound probe is measured; the position and attitudeof the biopsy needle is measured; the positions and attitudes of theultrasound beam and the tip of the biopsy needle are calculated; and theposition of the biopsy needle is displayed on the image provided by theultrasound scanner.
 11. A method substantially as described.